Consensus-Based Techniques for Range-Task Resolution in Crowdsourcing Systems
نویسندگان
چکیده
In crowdsourcing, a range task is a type of creation task where only free answers belonging to the numeric domain are accepted/possible. In this paper, we present the median-onagreement (ma) techniques based on statistical and consensusbased mechanisms for determining the result of range tasks. The ma techniques are characterized by i) the distinction between the group of workers that agree (i.e., workers in the consensus) on the task result from the group that disagree, and ii) the calculation of the final task answer through a median-based mechanism where only answers of workers in the consensus are considered.
منابع مشابه
Perform Three Data Mining Tasks with Crowdsourcing Process
For data mining studies, because of the complexity of doing feature selection process in tasks by hand, we need to send some of labeling to the workers with crowdsourcing activities. The process of outsourcing data mining tasks to users is often handled by software systems without enough knowledge of the age or geography of the users' residence. Uncertainty about the performance of virtual user...
متن کاملCorpus based coreference resolution for Farsi text
"Coreference resolution" or "finding all expressions that refer to the same entity" in a text, is one of the important requirements in natural language processing. Two words are coreference when both refer to a single entity in the text or the real world. So the main task of coreference resolution systems is to identify terms that refer to a unique entity. A coreference resolution tool could be...
متن کاملIncentives for truthful reporting in crowdsourcing
A challenge with the programmatic access of human talent via crowdsourcing platforms is the specification of incentives and the checking of the quality of contributions. Methodologies for checking quality include providing a payment if the work is approved by the task owner and hiring additional workers to evaluate contributors’ work. Both of these approaches place a burden on people and on the...
متن کاملSelection and aggregation techniques for crowdsourced semantic annotation task
Crowdsourcing is an accessible and cost-effective alternative to traditional methods of collecting and annotating data. The application of crowdsourcing to simple tasks has been well investigated. However, complex tasks like semantic annotation transfer require workers to take simultaneous decisions on chunk segmentation and labeling while acquiring on-the-go domainspecific knowledge. The incre...
متن کاملEnabling Physical Crowdsourcing On-the-Go with Context-Sensitive Notifications
This paper introduces the idea of sending timely notifications to potential helpers to complete small tasks onthe-go with minimal effort to them and maximal benefit to the system. We present two on-the-go crowdsourcing systems: Libero for package delivery, and CrowdFound for finding lost items. To encourage contributions, we introduce notification techniques that present task opportunities when...
متن کامل